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Executive Summary 



T his report provides a practical discussion of what is required to develop 
a school district "management guide," along with an actual guide 
built on evidence-based indicators. It begins with an imaginary discus- 
sion at Rebel Valley School District, during which a new superintendent 
leads his board through the guide. 

Indicators are discrete pieces of information, like water temperature or the 
Dow Jones Average, designed to alert leaders and members of the public 
about what is going on in large, complex systems. They provide warnings 
and hints about how well complex systems are functioning. They are, 
therefore, capable of alerting leaders to potential problems. Although they 
can help identify problems, they cannot provide solutions. 

To be effective, indicators need to be very powerful in terms of the quality 
of data, the utility of the information they provide, and their ability to com- 
municate something important and meaningful. 

The seven indicators of interest in the management system described in 
this report are: 

• Achievement (reading and mathematics). 

• Elimination of the achievement gap. 

• Student attraction (school ability to attract students). 

• Student engagement with the school. 

• Student retention/completion. 

• Teacher attraction and retention. 

• Funding equity. 

Several implications flow from the analysis contained in this report: 

Less may be more. School systems are now awash in data and information. 
The human capacity to absorb information is of necessity limited. Indicator 
systems should respect that reality. 

The principles of parsimony and power should be respected. The 

temptation to develop 1 7 indicators, or even 1 27 different pieces of informa- 
tion capable of satisfying everyone with a question about anything in every 
individual school, must be avoided The key to success will lie in parsimoni- 
ously selecting a few indicators and judging them against the standards of 
data, proxy, and communications power. 
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Executive Summary 



Current status data is necessary but not sufficient. Without a per- 
spective grounded over time, the public may be confused by year-to-year 
pronouncements about how well (or poorly) things are going. 

Smart use of data holds the potential of dramatically altering the 
tone and quality of board-superintendent relationships. Data sets 
that identify problems and promise to "get at" real issues on a school-by- 
school basis offer district leaders what all of them want — the opportunity to 
target scarce resources where they can do the most good. 

Targeting resources where they can do the most good requires 
better funding indicators. If district leaders genuinely seek strategic use 
of limited resources, it is essential that they stop "resourcing" schools and 
start thinking about the real dollar amounts spent in each of them. 

Currently, teacher attraction and retention are the best proxies 
we have for teacher effectiveness. Reliable indicators on teacher and 
principal quality are hard to come by. The lack of such indicators greatly 
hampers our ability to measure the impact of teachers and principals in any 
given school. 

The seven school-level indicators discussed in this paper are a solid 
jumping off point for any district. Although these indicators are well 
grounded in research and experience, each district will have to decide for 
itself the extent to which any (or all) of these seven measures fit its particular 
needs and circumstances. 

Professional development and technical assistance will be re- 
quired. In recent years, leadership sophistication about data usage has 
increased dramatically. Still, effective use of data as a management tool 
will undoubtedly require additional professional development or technical 
assistance. 

State leaders have a significant role to play. The role of state leaders 
becomes the role of leadership everywhere: pointing people in the right 
direction, providing political cover, and helping districts move along. 

In many ways, indicator development moves beyond bottom-line assessment 
systems to encourage new ways of thinking about accountability, while do- 
ing the right thing and taking the time to do it right. The work outlined in 
this report suggests that educators and leaders can find a better way, and 
then provides a concrete example of how this might be done. 
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Preface 



// r'Nepend upon it, sir," said Samuel Johnson, one of the great wits 
L^of the eighteenth century, "when a man knows he is to be 
hanged in a fortnight, it concentrates his mind wonderfully." Johnson 
would have had no trouble understanding the recent response of school 
districts across the country to the many government carrots and sticks 
before them. The accountability provisions of enactments such as the 
No Child Left Behind Act (NCLB) have concentrated education's mind 
wonderfully. 

Now that federal money is tied to "adeguate yearly progress" and parents 
can transfer their children from a "failing" school, there is major focus on 
guantitative rating of school and district level success. This is revolutionary. 
Instead of concentrating primarily on the "inputs" into the school system 
(dollars, classroom size, classes offered, size of library, number and guality 
of teachers), school leaders are now focusing on the "outputs": whether 
students learn when they are in the classroom and whether they leave the 
classroom at the appropriate time and with the appropriate credentials in 
hand. These kinds of data are now being used to make judgments about 
the value of individual schools and districts, and there is substantial criticism 
of this new reality. 

What is often overlooked is that the "outputs" can also be used by system 
leaders to make decisions about where they should intervene and what they 
should emphasize as they seek to assist struggling schools and students in 
them. In other words, information that some critics think is only used in an 
arbitrary way to judge schools can, instead, be used to help leaders make 
critical decisions about what to do and when to do it. 

Accountability, in short, can be diagnostic and constructive; it does not have 
to be punitive or destructive. 

One challenge is that school leaders are overwhelmed by educational minu- 
tiae in this new environment. There is probably treasure in there somewhere, 
but, buried as it is in mountains of ill-understood data, it is hard to discern 
the shape or potential value of these gems. While there is undoubted interest 
in gathering and reporting both input and output data on schools, much of 
what is on district websites and district-produced school guides is scattershot 
and unfocused. Numbers collected and presented in this way provide masses 
of data but little guidance about what the information means, how to use 
it, or what to do with it. The result, as one study of district data needs and 
uses puts it, is that "Most urban cities lack the strategic information to suc- 
cessfully identify and implement a district reform strategy ." 1 
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This report sets out to define and construct a working model of a school 
management guide that will help school officials, school leaders, and com- 
munity members make sense of the mountains of data they are generating. 
It is not so much a map to buried treasure as it is a guide to action. It does 
not say "X marks the spot," but it does provide some general guidance on 
where to start digging and how to begin. 

This guide grew from an extensive study of school district leaders and 
how they use data in making decisions. It is informed by analyses of how 
leaders in other areas of community life try to understand how their public 
institutions measure up against public institutions elsewhere. It is designed 
to assist leaders — of individual schools, school districts, and state education 
systems — improve school management. One way it attempts to do this is 
by describing how leaders can use key indicators to understand and use 
the data they already have available. Through carefully selected indicators, 
school leaders can examine how well their schools are doing in relation to 
comparable schools elsewhere, as well as how things are changing over 
time. The foundation of the guide is work completed over the last six years 
at the University of Washington's Center on Reinventing Public Education 
(CRPE), much of it supported by The Wallace Foundation. This work fo- 
cused on the achievement gap as measured by Washington Assessment of 
Student Learning (WASL) — the state's standardized testing system, national 
and statewide dropout statistics, national studies of superintendents and 
principals, and extensive work on school and school district reform, includ- 
ing school finance. 

At the heart of the management guide is a new educational indicator system 
designed to provide focus without adding new layers of data collection and 
analysis. Indicators are discrete pieces of information that tell us something 
about what is going on in a larger system. More technically, an indicator 
is "simply a set of rules of gathering and organizing data so they can be 
assigned meaning ." 2 Indicators are often single items or indices of data 
that provide information about an underlying characteristic. The readings 
on automobile speedometers and gas gauges are indicators. A fever ther- 
mometer reading is an indicator. New factory orders and housing starts are 
indicators, as are rates of unemployment and hospital morbidity. The point 
is that whenever we are unable to view a large system in its totality — whether 
an automobile, the human body, the national economy, a local community, 
a school system, or a hospital — indicators can provide a general sense of 
how well the system is functioning. The trick is to find indicators that both 
have meaning and are easy to read. If the engine is overheating, the driver 
needs to be able to tell that at a glance. 

In the case of education, the reguirement that indicators have meaning and 
be easy to read pre-supposes that measures have been validated by research 
as related to student learning. Perhaps even more difficult where educators 
are concerned, it pre-supposes that indicators are capable of being presented 
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in a comprehensible graph or chart, or presented in a page or two at most, 
rather than in a volume. 

Many school leaders have the skill to sift through reams of data to see how 
the system is performing (as a whole and in its parts), yet they seldom have 
the time or the training to do so. Those outside district leadership — prin- 
cipals, teachers, parents, and the public — generally have neither the time, 
the skill, nor the inclination. Two unattractive developments potentially 
arise from this state of affairs. The first is that, faced with piles of confusing 
and sometimes contradictory data, all shrug and throw up their hands. The 
second is that one or two discrete and readily understandable pieces of data 
will be seized upon as definitive evidence that the schools are "failing" or, 
alternatively, "turning the corner." The management guide presented here 
provides indicators that respect both the complexities of the institution they 
are describing and the subtleties of the data, while being meaningful and 
straightforward. 

This report is the fifth developed at the Center on Reinventing Public Educa- 
tion for The Wallace Foundation. Earlier, the Center produced: 

• A Matter of Definition: Is There Truly a Shortage of School 
Principals? (Marguerite Roza et al., January 2003.) 

• An Impossible job?: The View from the Urban Superintendent's 
Chair. (Howard Fuller et al., July 2003.) 

• Making Sense of Leading Schools: A Study of the School 
Principalship. (Bradley Portin et al., September 2003.) 

• From Bystander to Ally: Transforming the District Human 
Resources Department. (Christine Campbell et al., April 2004.) 

Buried Treasure is divided into four chapters. Following the summary and 
this preface, the first chapter provides an example of how the management 
guide would look and could be used in a typical (i.e., highly complex) urban 
school district. It does so by reproducing an imaginary transcript of a school 
district board meeting where the management guide is introduced. The 
second chapter provides a short review of indicator systems, a description of 
the criteria used for selection of the indicators used here, and a commentary 
on the potential sources of the data. Chapter 3 provides a detailed expla- 
nation of each element in the guide, discussing the reasons for using the 
particular indicator and how it would be displayed for individual schools in 
a district. These charts and graphs could, potentially, be used as part of an 
individual school report card that is then aggregated into a district report 
card. Chapter 4 concludes with implications for state and district leaders. 

In providing an example of how the guide would look and could be used 
in practice — at both the individual school and district levels — the guide is 
unique among indicator systems presently available. In most cases, the lit- 
erature provides only prescription (i.e., telling leaders what data they should 
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